/LMAR=0/XLINE=3/FONT#0=BASL30/FONT#1=BASI30/FONT#2=BASB30



␈↓ α∧␈↓␈↓ ε[(draft)
␈↓ α∧␈↓␈↓ ∧ASCRIBING MENTAL QUALITIES TO MACHINES

␈↓ α∧␈↓        To␈α   ascribe␈α   certain␈α   ␈↓↓beliefs␈↓,␈α    ␈↓↓knowledge␈↓,␈α␈↓↓intentions␈↓,␈α ␈↓↓consciousness␈↓,␈α ␈↓↓abilities␈↓␈α or␈α␈↓↓wants␈↓
␈↓ α∧␈↓to␈αa␈αmachine␈αor␈αcomputer␈αprogram␈αis␈α␈↓αlegitimate␈↓␈α whenever␈α such␈α ascription␈αexpresses␈α the␈α
 same
␈↓ α∧␈↓information␈α⊃ about␈α⊃ the␈α⊂machine␈α⊃as␈α⊃it␈α⊃expresses␈α⊂about␈α⊃a␈α⊃person.␈α⊂    It␈α⊃ is␈α⊃ ␈↓αuseful␈↓␈α⊃ when␈α⊂ the
␈↓ α∧␈↓ascription␈α helps␈α us␈αunderstand␈αthe␈αstructure␈αof␈αthe␈αmachine,␈αits␈αpast␈αor␈αfuture␈αbehavior,␈αor␈αhow
␈↓ α∧␈↓to␈α⊂repair␈α⊃or␈α⊂improve␈α⊃it.␈α⊂ Theories␈α⊃of␈α⊂belief,␈α⊂ knowledge␈α⊃ and␈α⊂wanting␈α⊃can␈α⊂be␈α⊃constructed␈α⊂for
␈↓ α∧␈↓machines␈α⊂in␈α∂a␈α⊂simpler␈α⊂setting␈α∂than␈α⊂for␈α⊂humans␈α∂and␈α⊂later␈α⊂applied␈α∂to␈α⊂humans.␈α⊂   Ascription␈α∂of
␈↓ α∧␈↓mental␈α
qualities␈α
is␈α
␈↓αmost␈α
straightforward␈↓␈α
for␈α
machines␈α
of␈α
known␈α
structure␈α
such␈α
as␈α
thermostats␈α
and
␈↓ α∧␈↓computer␈αoperating␈αsystems,␈αbut␈αis␈α␈↓αmost␈αuseful␈↓␈α when␈αapplied␈α to␈α machines␈α whose␈α structure␈α is
␈↓ α∧␈↓very incompletely known.  General arguments for these assertions will follow examples.

␈↓ α∧␈↓        As␈α∂ much␈α∞ as␈α∂ possible,␈α∞ we␈α∂ will␈α∞ ascribe␈α∂ mental␈α∞ qualities␈α∂separately␈α∞ from␈α∂ each␈α∞ other
␈↓ α∧␈↓instead␈αof␈αbundling␈α
them␈αtogether␈αinto␈α
a␈αconcept␈αof␈α
mind.␈α This␈αis␈α
necessary␈αin␈αtreating␈α
machines,
␈↓ α∧␈↓because␈α∃present␈α∀ machines␈α∃ have␈α∀ rather␈α∃varied␈α∃little␈α∀minds;␈α∃the␈α∀mental␈α∃qualities␈α∃that␈α∀can
␈↓ α∧␈↓legitimately␈αbe␈αascribed␈αto␈αthem␈αare␈αfew␈αand␈αdiffer␈αfrom␈αmachine␈αto␈αmachine.␈α After␈αall,␈αwe␈αcan't
␈↓ α∧␈↓say␈αwhat␈αcollection␈αof␈αmental␈αqualities␈αconstitutes␈αa␈αmind␈αfor␈αhumans.␈α Thus␈αwe␈αwill␈αnot␈αeven␈αtry
␈↓ α∧␈↓to␈α⊂meet␈α⊂objections␈α⊂like,␈α⊂␈↓↓"Unless␈α∂it␈α⊂also␈α⊂does␈α⊂X,␈α⊂it␈α⊂is␈α∂illegitimate␈α⊂to␈α⊂speak␈α⊂of␈α⊂its␈α⊂having␈α∂mental
␈↓ α∧␈↓↓qualities."␈↓

␈↓ α∧␈↓        Mental␈α⊗qualities␈α∃differ␈α⊗ in␈α∃their␈α⊗ prevalence.␈α∃ For␈α⊗ example,␈α∃machines␈α⊗as␈α⊗ simple␈α∃as
␈↓ α∧␈↓thermostats␈αcan␈α be␈αsaid␈αto␈αhave␈α beliefs,␈αand␈αhaving␈αbeliefs␈αseems␈αto␈αbe␈αa␈αcharacteristic␈α of␈αmost
␈↓ α∧␈↓machines␈α
capable␈α
of␈α∞problem␈α
solving␈α
performance.␈α
 However,␈α∞the␈α
machines␈α
mankind␈α
has␈α∞so␈α
far
␈↓ α∧␈↓found␈αit␈αuseful␈αto␈α
construct␈αrarely␈αhave␈αbeliefs␈αabout␈α
beliefs.␈α (Beliefs␈αabout␈αbeliefs␈α
are␈αnecessary
␈↓ α∧␈↓for␈αfuture␈αcomputer␈αprograms␈αthat␈αmust␈αreason␈αabout␈αwhat␈αknowledge␈αthey␈αlack␈αand␈αwhere␈αto␈αget
␈↓ α∧␈↓it.)␈α⊃Other␈α∩mental␈α⊃qualities,␈α⊃such␈α∩as␈α⊃love␈α∩and␈α⊃hate,␈α⊃seem␈α∩peculiar␈α⊃to␈α∩human-like␈α⊃motivational
␈↓ α∧␈↓structures␈αand␈αwill␈α not␈αbe␈α
required␈α for␈αuseful␈αintelligent␈αmachines␈α
 per␈αse,␈αbut␈αwe␈αcould␈α
program
␈↓ α∧␈↓them␈α∞if␈α∞we␈α∞wanted␈α∂ to,␈α∞because␈α∞ our␈α∞ common␈α∞sense␈α∂ notions␈α∞about␈α∞ them␈α∞translate␈α∂readily␈α∞into
␈↓ α∧␈↓certain␈αprogram␈αand␈α
data␈αstructures.␈α Still␈α
other␈αmental␈αqualities,␈α
e.g.␈αhumor␈αand␈α
appreciation␈αof
␈↓ α∧␈↓beauty,␈α
seem␈α
much␈α
harder␈αto␈α
model.␈α
 While␈α
we␈α
are␈αquite␈α
liberal␈α
in␈α
ascribing␈α
some␈αmental␈α
qualities
␈↓ α∧␈↓even␈α
to␈α
rather␈α
primitive␈α
machines,␈α
we␈α
are␈αtrying␈α
to␈α
be␈α
conservative␈α
in␈α
our␈α
criteria␈α
for␈αascribing
␈↓ α∧␈↓particular qualities.

␈↓ α∧␈↓        The␈α∞motivation␈α
for␈α∞this␈α
work␈α∞comes␈α∞from␈α
artificial␈α∞intelligence␈α
which␈α∞raises␈α∞for␈α
machines
␈↓ α∧␈↓issues␈α
that␈α
have␈α
previously␈αbeen␈α
considered␈α
only␈α
in␈α
connection␈αwith␈α
people.␈α
 To␈α
give␈αone␈α
example,
␈↓ α∧␈↓in␈α⊂order␈α⊂to␈α⊂behave␈α∂intelligently,␈α⊂a␈α⊂machine␈α⊂must␈α∂sometimes␈α⊂say,␈α⊂␈↓↓"I␈α⊂can,␈α∂but␈α⊂I␈α⊂won't"␈↓.␈α⊂ We␈α∂are
␈↓ α∧␈↓therefore led to study the conditions under which a machine can correctly say this.

␈↓ α∧␈↓        In␈α∂order␈α∂to␈α∂ascribe␈α∂mental␈α∞qualities␈α∂to␈α∂machines,␈α∂we␈α∂will␈α∞use␈α∂what␈α∂ we␈α∂call␈α∂ ␈↓↓second␈α∞order
␈↓ α∧␈↓↓structural␈α
definitions␈↓.␈α
  Structural␈α
definitions␈α
of␈α
 qualities␈α
are␈α
 given␈α
in␈α
 terms␈α
of␈α
 the␈α
 state␈α
of␈α
 the
␈↓ α∧␈↓system␈α
being␈α
described␈α
rather␈α
than␈α
in␈α
terms␈α
of␈α
its␈α
behavior.␈α
 The␈α
most␈α
 straightforward␈α
 structural
␈↓ α∧␈↓definitions␈α
 are␈αfirst␈α
 order,␈α
but␈αunfortunately,␈α
 for␈α
 many␈α concepts,␈α
 they␈α
 may␈α not␈α
  exist␈α  in␈α
  a
␈↓ α∧␈↓satisfactory␈αform.␈α
 While␈α second␈αorder␈α
 definitions␈αof␈α mental␈α
qualities␈α may␈αbe␈α
 new␈αas␈α a␈α
formal
␈↓ α∧␈↓device,␈α∞ we␈α∞ think␈α∞ they␈α∞correspond␈α∞to␈α∞ common␈α∞ways␈α∞ of␈α∞introducing␈α∞ new␈α∞concepts␈α∞in␈α∞ science,
␈↓ α∧␈↓philosophical logic and everyday life.



␈↓ α∧␈↓        Behavioral␈α definitions␈αare␈αoften␈α favored␈αin␈α philosophy.␈α A␈αsystem␈αis␈α defined␈αto␈α have␈α a
␈↓ α∧␈↓certain␈α∞ quality␈α∞if␈α∞ it␈α∞behaves␈α∞ in␈α∞ a␈α∞certain␈α∞way␈α∞ or␈α∞ is␈α∞ ␈↓↓disposed␈↓␈α∞to␈α∞behave␈α∞ in␈α∞a␈α∂ certain␈α∞way.
␈↓ α∧␈↓Their␈α∩ virtue␈α⊃is␈α∩that␈α⊃ they␈α∩don't␈α⊃postulate␈α∩internal␈α⊃ states␈α∩that␈α⊃are␈α∩unobservable␈α∩ to␈α⊃ present
␈↓ α∧␈↓science␈α
 and␈α
 may␈α
 remain␈α
  unobservable.␈α∞ However,␈α
such␈α
 definitions␈α
are␈α
awkward␈α∞for␈α
 mental
␈↓ α∧␈↓qualities,␈αbecause␈α
common␈αsense␈α suggests␈α
 that␈α a␈α
 mental␈αquality␈α may␈α
 not␈α result␈α
 in␈αbehavior,
␈↓ α∧␈↓because␈α⊂another␈α⊃mental␈α⊂ quality␈α⊂may␈α⊃prevent␈α⊂it,␈α⊂e.g.␈α⊃ I␈α⊂may␈α⊂think␈α⊃you␈α⊂are␈α⊃ thick-headed,␈α⊂but
␈↓ α∧␈↓politeness␈α∀may␈α∃prevent␈α∀my␈α∀saying␈α∃so.␈α∀ Particular␈α∀ difficulties␈α∃ can␈α∀ be␈α∀ overcome,␈α∃ but␈α∀ an
␈↓ α∧␈↓impression␈α of␈αvagueness␈αremains.␈α The␈αliking␈α for␈αbehavioral␈αdefinitions␈αstems␈αfrom␈αcaution,␈αbut
␈↓ α∧␈↓I␈αwould␈αinterpret␈αscientific␈αexperience␈αas␈αshowing␈αthat␈αboldness␈αin␈αpostulating␈α complex␈αstructures
␈↓ α∧␈↓of␈αunobserved␈α entities␈α
-␈αprovided␈αit␈αis␈α
 accompanied␈αby␈αa␈αwillingness␈α
to␈α take␈αback␈αmistakes␈α
-␈αis
␈↓ α∧␈↓more␈α⊃ likely␈α∩ to␈α⊃ be␈α⊃ rewarded␈α∩by␈α⊃ control␈α⊃ over␈α∩ nature␈α⊃ than␈α⊃ is␈α∩positivistic␈α⊃timidity.␈α∩  It␈α⊃is
␈↓ α∧␈↓particularly␈α∞instructive␈α∞to␈α∞ imagine␈α∞a␈α∞determined␈α∞ behaviorist␈α∞trying␈α∞to␈α∞figure␈α∞ out␈α∞an␈α
electronic
␈↓ α∧␈↓computer.␈α_ Trying␈α→to␈α_ define␈α_each␈α→quality␈α_ behaviorally␈α_would␈α→ get␈α_him␈α→ nowhere;␈α_only
␈↓ α∧␈↓simultaneously␈α∃postulating␈α⊗a␈α∃complex␈α∃structure␈α⊗including␈α∃memory,␈α∃arithmetic␈α⊗ unit,␈α∃ control
␈↓ α∧␈↓structure, and  input-output  would yield predictions that could be compared with experiment.

␈↓ α∧␈↓        In␈α⊃the␈α⊃case␈α∩of␈α⊃a␈α⊃specific␈α⊃known␈α∩machine,␈α⊃one␈α⊃can␈α⊃often␈α∩give␈α⊃a␈α⊃␈↓↓first␈α∩ order␈α⊃ structural
␈↓ α∧␈↓↓definition␈↓.␈α
   Thus␈α
 we␈α
 might␈α
 give␈α a␈α
predicate␈α
␈↓↓B(s,p)␈↓␈α
such␈α
that␈α
if␈αthe␈α
 machine␈α
is␈α
in␈α
state␈α
␈↓↓s␈↓,␈α it␈α
is
␈↓ α∧␈↓said␈αto␈α
 believe␈αthe␈α
sentence␈α ␈↓↓p␈↓␈αprovided␈α
␈↓↓B(s,p)␈↓␈α is␈α
true.␈α However,␈αI␈α
don't␈αthink␈α
there␈αis␈αa␈α
general
␈↓ α∧␈↓definition of belief having this form that applies to all machines in all environments.

␈↓ α∧␈↓        Instead␈α
we␈αgive␈α
a␈α
␈↓↓second␈αorder␈α
predicate␈↓␈α β␈↓↓(B,W)␈↓␈α
that␈α
tells␈αus␈α
whether␈αwe␈α
regard␈α
the␈α first
␈↓ α∧␈↓order␈α
predicate␈α
␈↓↓B(s,p)␈↓␈α
as␈α
a␈α
"good"␈α
notion␈α
 of␈α
belief␈α
in␈α
the␈α
␈↓↓world␈α
W␈↓.␈α
 Such␈α
a␈α
predicate␈α
β␈α
will␈αbe
␈↓ α∧␈↓called␈αa␈α ␈↓↓second␈α order␈α definition␈↓;␈αit␈α gives␈α criteria␈α for␈αcriticizing␈α  an␈α ascription␈α of␈α  a␈α quality
␈↓ α∧␈↓to␈α∂ a␈α∂ system.␈α∂ Axiomatizations␈α∞of␈α∂ belief␈α∂ give␈α∂  rise␈α∂ to␈α∞ a␈α∂ second␈α∂  order␈α∂definitions,␈α∂and␈α∞we
␈↓ α∧␈↓suggest␈α⊃that␈α⊃both␈α⊃our␈α⊃ common␈α⊃sense␈α⊃and␈α⊃scientific␈α⊃usage␈α⊃ of␈α⊃not-directly-observable␈α⊂qualities
␈↓ α∧␈↓corresponds␈α⊃more␈α⊃closely␈α⊂to␈α⊃second␈α⊃ order␈α⊃structural␈α⊂ definition␈α⊃than␈α⊃ to␈α⊃ordinary␈α⊂ behavioral
␈↓ α∧␈↓definition.␈α  It␈α should␈α be␈α noted␈αthat␈α a␈α second␈αorder␈α definition␈αdoesn't␈α guarantee␈α that␈α there
␈↓ α∧␈↓exist␈α
 predicates␈α∞ ␈↓↓B␈↓␈α
 meeting␈α
the␈α∞criterion␈α
β␈α
or␈α∞ that␈α
␈↓↓B␈↓␈α
is␈α∞unique.␈α
  It␈α
can␈α∞ also␈α
turn␈α
out␈α∞that␈α
 a
␈↓ α∧␈↓quality  can best  be defined as part of a group of interacting qualities.

␈↓ α∧␈↓        Our␈α
 second␈α  order␈α
  definition␈α criticizes␈α
  whole␈α
  belief␈αstructures␈α
rather␈αthan␈α
 individual
␈↓ α∧␈↓beliefs.␈α⊃ We␈α⊃can␈α⊃ treat␈α⊃individual␈α⊃beliefs␈α⊃ by␈α⊃ saying␈α⊃ that␈α⊃a␈α⊃ system␈α⊃ believes␈α⊃ ␈↓↓p␈↓␈α⊃ in␈α⊃state␈α⊃ ␈↓↓s␈↓
␈↓ α∧␈↓provided␈αall␈α"reasonably␈αgood"␈α ␈↓↓B␈↓'s␈α
satisfy␈α␈↓↓B(s,p)␈↓.␈α Thus␈α we␈αare␈αdistinguishing␈α
the␈α"intersection"
␈↓ α∧␈↓of the reasonably good ␈↓↓B␈↓'s.

␈↓ α∧␈↓        It␈α
 seems␈α
 to␈α
 me␈α
 that␈α
 there␈αshould␈α
 be␈α
 a␈α
 metatheorem␈α
 of␈α
mathematical␈α
 logic␈αasserting
␈↓ α∧␈↓that␈α∞not␈α∞all␈α∞ second␈α∞order␈α∞definitions␈α∂can␈α∞ be␈α∞reduced␈α∞ to␈α∞ first␈α∞ order␈α∞definitions␈α∂ and␈α∞ further
␈↓ α∧␈↓theorems␈α∂characterizing␈α∂ those␈α∂ second␈α⊂  order␈α∂ definitions␈α∂ that␈α∂ admit␈α∂ such␈α⊂reductions.␈α∂ Such
␈↓ α∧␈↓technical␈α
results,␈α
if␈α
 they␈α
can␈α
be␈α
 found,␈α
may␈α
 be␈α
helpful␈α
in␈α
 philosophy␈α
and␈α
in␈α
 the␈α
construction␈α
of
␈↓ α∧␈↓formal␈αscientific␈αtheories.␈α I␈αwould␈αconjecture␈αthat␈αmany␈αof␈αthe␈αinformal␈αphilosophical␈αarguments
␈↓ α∧␈↓that␈α∩certain␈α∩mental␈α∩ concepts␈α∪cannot␈α∩be␈α∩reduced␈α∩to␈α∩ physics␈α∪will␈α∩turn␈α∩out␈α∩to␈α∩be␈α∪sketches␈α∩of
␈↓ α∧␈↓arguments that these concepts require second (or higher) order definitions.

␈↓ α∧␈↓        Here␈α
 is␈α
how␈α
second␈α
order␈α
definition␈α
works␈α
in␈α
the␈α
case␈α
of␈α
 belief.␈α
 For␈α
each␈α
state␈α
␈↓↓s␈↓␈α
of␈αthe
␈↓ α∧␈↓machine␈αand␈α
each␈α sentence␈α
␈↓↓p␈↓␈αin␈α a␈α
suitable␈α language␈α
␈↓↓L␈↓,␈αa␈αbelief␈α
predicate␈α␈↓↓B(s,p)␈↓␈α
assigns␈α truth
␈↓ α∧␈↓or␈α∞falsity␈α∞according␈α∞to␈α∞whether␈α∞the␈α∞machine␈α∞is␈α∂considered␈α∞to␈α∞believe␈α∞␈↓↓p␈↓␈α∞ when␈α∞it␈α∞ is␈α∞ in␈α∂state␈α∞ ␈↓↓s␈↓.



␈↓ α∧␈↓The␈α language␈α ␈↓↓L␈↓␈αis␈α chosen␈α
 for␈α our␈αconvenience,␈αand␈α there␈α
is␈αno␈αassumption␈α that␈αthe␈α
machine
␈↓ α∧␈↓explicitly␈α
represents␈α
sentences␈α
of␈α
␈↓↓L␈↓␈α∞in␈α
any␈α
way.␈α
 Thus␈α
we␈α∞can␈α
talk␈α
about␈α
the␈α
beliefs␈α∞of␈α
Chinese,
␈↓ α∧␈↓dogs,␈α∞thermostats,␈α∂and␈α∞computer␈α∂operating␈α∞systems␈α∞without␈α∂ assuming␈α∞that␈α∂they␈α∞ use␈α∂English␈α∞or
␈↓ α∧␈↓our  favorite first order language.

␈↓ α∧␈↓        Here are some criteria we may want ␈↓↓B(s,p)␈↓ to satisfy:

␈↓ α∧␈↓        1.␈α
The␈α∞set␈α
␈↓↓Bel(s)␈↓␈α
of␈α∞beliefs,␈α
i.e.␈α
the␈α∞set␈α
of␈α
␈↓↓p␈↓'s␈α∞for␈α
which␈α
 ␈↓↓B(s,p)␈↓␈α∞is␈α
 assigned␈α∞true␈α
 contains
␈↓ α∧␈↓sufficiently "obvious" consequences of some of its members.

␈↓ α∧␈↓        2.␈α  ␈↓↓Bel(s)␈↓␈αchanges␈α in␈αa␈α reasonable␈αway␈α when␈αthe␈α state␈αchanges.␈α   We␈α like␈α new␈α beliefs
␈↓ α∧␈↓to␈α  be␈α logical␈α or␈α "plausible"␈αconsequences␈αof␈αold␈α ones␈αor␈αto␈αcome␈αin␈αas␈α␈↓↓communications␈↓␈αin␈αsome
␈↓ α∧␈↓language␈α∀on␈α∀the␈α∀input␈α∀lines␈α∀ or␈α∀to␈α∪be␈α∀␈↓↓observations␈↓,␈α∀i.e.␈α∀ beliefs␈α∀about␈α∀the␈α∀environment␈α∪the
␈↓ α∧␈↓information␈α∂for␈α∂which␈α⊂comes␈α∂in␈α∂on␈α∂the␈α⊂input␈α∂lines.␈α∂ The␈α∂set␈α⊂of␈α∂beliefs␈α∂should␈α∂not␈α⊂change␈α∂too
␈↓ α∧␈↓rapidly as the state changes.

␈↓ α∧␈↓        3.␈α  We␈αprefer␈α the␈αset␈α of␈αbeliefs␈α to␈αbe␈α as␈αconsistent␈α as␈αpossible.␈α (Admittedly,␈αconsistency
␈↓ α∧␈↓is␈αnot␈αa␈αquantitative␈αconcept␈αin␈αmathematical␈αlogic␈α-␈α a␈αsystem␈αis␈α either␈αconsistent␈αor␈α not,␈αbut␈α it
␈↓ α∧␈↓would␈α
seem␈α
that␈α
we␈α
 will␈αsometimes␈α
have␈α
to␈α
ascribe␈α
 inconsistent␈αsets␈α
of␈α
beliefs␈α
to␈α
machines␈αand
␈↓ α∧␈↓people.␈α Our␈αintuition␈αsays␈αthat␈αwe␈αshould␈αbe␈αable␈αto␈αmaintain␈αareas␈αof␈α consistency␈αin␈αour␈αbeliefs
␈↓ α∧␈↓and␈α∂that␈α∂ it␈α∂is␈α∞ especially␈α∂important␈α∂ to␈α∂avoid␈α∞ inconsistencies␈α∂in␈α∂ the␈α∂machine's␈α∂purely␈α∞analytic
␈↓ α∧␈↓beliefs).

␈↓ α∧␈↓        4.␈α Our␈αcriteria␈αfor␈αbelief␈αsystems␈αcan␈αbe␈αstrengthened␈αif␈αwe␈αidentify␈αsome␈αof␈αthe␈α machine's
␈↓ α∧␈↓beliefs␈αas␈αexpressing␈αgoals,␈α
 i.e.␈αif␈αwe␈αhave␈αbeliefs␈α
 of␈αthe␈αform␈α"It␈αwould␈α
be␈αgood␈αif␈α ...".␈α Then␈α
we
␈↓ α∧␈↓can␈α⊂ask␈α⊂that␈α⊂the␈α⊂machine's␈α⊂behavior␈α⊂ be␈α∂somewhat␈α⊂␈↓↓rational␈↓,␈α⊂i.e.␈α⊂ ␈↓↓it␈α⊂does␈α⊂what␈α⊂ it␈α⊂ believes␈α∂will
␈↓ α∧␈↓↓achieve␈α
its␈α goals␈↓.␈α
The␈α
 more␈αof␈α
 its␈αbehavior␈α
 we␈α
can␈α account␈α
for␈αin␈α
 this␈α
way,␈α the␈α
better␈αwe␈α
 will
␈↓ α∧␈↓like␈α␈↓↓B(s,p)␈↓.␈α We␈α
also␈αwould␈αlike␈α
to␈αaccount␈αfor␈α
internal␈αstate␈αchanges␈α
as␈αchanges␈αin␈α
belief␈αin␈αso␈α
far
␈↓ α∧␈↓as this is reasonable.

␈↓ α∧␈↓        5.␈α∩ If␈α∪the␈α∩machine␈α∪communicates,␈α∩i.e.␈α∪emits␈α∩sentences␈α∪in␈α∩some␈α∪language␈α∩ that␈α∪ can␈α∩ be
␈↓ α∧␈↓interpreted␈α as␈α assertions,␈α questions␈α
 and␈αcommands,␈αwe␈αwill␈αwant␈α
the␈αassertions␈αto␈αbe␈αamong␈α
 its
␈↓ α∧␈↓beliefs␈α⊂unless␈α⊃we␈α⊂are␈α⊂ ascribing␈α⊃to␈α⊂it␈α⊃ a␈α⊂goal␈α⊂or␈α⊃ subgoal␈α⊂that␈α⊂involves␈α⊃ lying.␈α⊂ In␈α⊃general,␈α⊂its
␈↓ α∧␈↓communications  should  be such  as  it  believes  will achieve its goals.

␈↓ α∧␈↓        6.␈α∂ Sometimes␈α∂we␈α∂shall␈α∂want␈α∞to␈α∂ascribe␈α∂introspective␈α∂beliefs,␈α∂e.g.␈α∞a␈α∂belief␈α∂that␈α∂it␈α∂does␈α∞not
␈↓ α∧␈↓know how to fly to Boston or even that it doesn't know what it wants in a certain situation.

␈↓ α∧␈↓        7.␈α∂Finally,␈α∂we␈α∂will␈α∂prefer␈α∂a␈α∂more␈α∂economical␈α∂ascription␈α∂␈↓↓B␈↓␈α∂to␈α∂a␈α∂less␈α∂economical␈α∂one.␈α∞ The
␈↓ α∧␈↓fewer␈α
beliefs␈α
we␈α
ascribe␈α
and␈α
the␈α
less␈α
they␈α
change␈α
with␈α
change␈α
with␈α
state␈α
consistent␈α
with␈α
accounting
␈↓ α∧␈↓for the behavior and the internal state changes, the better we will like it.

␈↓ α∧␈↓        The␈α
above␈α
 criteria␈αhave␈α
 been␈α
 formulated␈α
somewhat␈α vaguely.␈α
 There␈α
would␈α
be␈α a␈α
definite
␈↓ α∧␈↓need␈α
to␈α
make␈α
them␈α
 more␈α
precise␈α
if␈α
it␈α
were␈α
to␈α
turn␈α
out␈α
that␈α
there␈α
 were␈α
widely␈α
different␈α
ascriptions
␈↓ α∧␈↓of␈α beliefs␈αto␈αa␈αparticular␈αmachine␈αthat␈α
all␈αmet␈αour␈α criteria␈αor␈αif␈αthe␈αcriteria␈α
allowed␈α ascriptions
␈↓ α∧␈↓that␈α
 differed␈α
widely␈α∞from␈α
 our␈α
intuitions.␈α∞  My␈α
present␈α
opinion␈α
is␈α∞that␈α
more␈α
thought␈α∞will␈α
make
␈↓ α∧␈↓the␈α criteria␈αsomewhat␈αmore␈αprecise␈α at␈αno␈αcost␈α in␈αapplicability,␈αbut␈αthat␈α they␈α␈↓↓should␈↓␈αstill␈αremain



␈↓ α∧␈↓rather␈α∞vague,␈α∞i.e.␈α∞we␈α
shall␈α∞want␈α∞to␈α∞ascribe␈α∞belief␈α
in␈α∞a␈α∞␈↓↓family␈↓␈α∞of␈α
cases.␈α∞ I␈α∞would␈α∞further␈α∞bet␈α
that
␈↓ α∧␈↓even␈α
at␈αthe␈α
present␈αlevel␈α
of␈αvagueness,␈α
there␈α
won't␈αbe␈α
radically␈αdifferent␈α
equally␈α"good"␈α
ascriptions
␈↓ α∧␈↓of␈α∩belief␈α∩for␈α∪systems␈α∩of␈α∩practical␈α∩interest.␈α∪ If␈α∩there␈α∩were,␈α∩we␈α∪would␈α∩notice␈α∩the␈α∪ambiguity␈α∩in
␈↓ α∧␈↓ourselves.␈α Even␈αso,␈αfor␈αparticular␈αclasses␈αof␈αmachines␈αand␈αparticular␈α  languages,␈α  we␈α   will␈α  be
␈↓ α∧␈↓interested   in   precise axiomatizations of β.




␈↓ α∧␈↓␈↓ ∧¬DEREDUCTION (THE OPPOSITE OF REDUCTION)

␈↓ α∧␈↓        ␈↓αWhy␈α∩ should␈α∩we␈α∩want␈α∩to␈α∩ ascribe␈α∪beliefs␈α∩to␈α∩ machines␈α∩ at␈α∩all?␈↓␈α∩This␈α∩is␈α∪the␈α∩opposite
␈↓ α∧␈↓question␈α∂to␈α∂that␈α∂involved␈α∂in␈α∂the␈α⊂controversy␈α∂over␈α∂␈↓↓reductionism␈↓.␈α∂ Instead␈α∂of␈α∂asking␈α⊂how␈α∂mental
␈↓ α∧␈↓qualities␈α⊂can␈α⊂be␈α⊂␈↓αreduced␈↓␈α⊂to␈α⊂physical␈α⊂ones,␈α∂we␈α⊂ask␈α⊂how␈α⊂to␈α⊂␈↓αascribe␈↓␈α⊂mental␈α⊂qualities␈α⊂to␈α∂physical
␈↓ α∧␈↓systems.␈α In␈αour␈αopinion,␈αthis␈αquestion␈αis␈αmore␈αnatural␈αand␈αwill␈αlead␈αto␈αgood␈αanswers␈αto␈αquestions
␈↓ α∧␈↓of reductionism.

␈↓ α∧␈↓        To␈α⊃ put␈α⊂the␈α⊃ issue␈α⊃sharply,␈α⊂ consider␈α⊃completely␈α⊂ deterministic␈α⊃ computer␈α⊃ programs␈α⊂ for
␈↓ α∧␈↓which␈α⊗ we␈α⊗ posess␈α⊗complete␈α⊗listings.␈α⊗ In␈α⊗this␈α⊗case,␈α⊗the␈α⊗behavior␈α⊗of␈α⊗the␈α⊗program␈α⊗ in␈α⊗any
␈↓ α∧␈↓environment␈α⊃is␈α⊃determined␈α⊃from␈α⊃the␈α⊃structure␈α⊃of␈α⊂the␈α⊃program␈α⊃and␈α⊃could␈α⊃be␈α⊃ found␈α⊃ out␈α⊂ by
␈↓ α∧␈↓simulating␈α
the␈α
 action␈α
 of␈α
 the␈α
 program␈α
 and␈α
the␈α
environment␈α
without␈α
having␈α
to␈α
deal␈α
with␈α
any
␈↓ α∧␈↓concept of belief.  There are several reasons for ascribing belief even in such cases:

␈↓ α∧␈↓        #.␈α
Although␈αwe␈α
may␈α
know␈αthe␈α
program,␈α
its␈αstate␈α
at␈αa␈α
given␈α
moment␈αis␈α
usually␈α
not␈αdirectly
␈↓ α∧␈↓observable,␈α∀and␈α∪the␈α∀conclusions␈α∪we␈α∀can␈α∀draw␈α∪about␈α∀its␈α∪current␈α∀state␈α∪may␈α∀be␈α∀more␈α∪readily
␈↓ α∧␈↓expressable as ascriptions of belief or want than in any other way.

␈↓ α∧␈↓        #.␈α
Even␈αif␈α
we␈αhave␈α
another␈αcomputer␈α
available␈αto␈α
simulate␈αthe␈α
interaction␈αof␈α
our␈αprogram
␈↓ α∧␈↓with␈α∩its␈α∩environment,␈α∩we␈α∩may␈α∩not␈α∩be␈α∩able␈α∩to␈α∩carry␈α∩out␈α∩the␈α∩simulation␈α∩fast␈α∩enough␈α∪to␈α∩ suit
␈↓ α∧␈↓ourselves.␈α∂Moreover,␈α∂we␈α∞may␈α∂not␈α∂have␈α∂the␈α∞initial␈α∂conditions␈α∂of␈α∂the␈α∞environment␈α∂or␈α∂its␈α∂laws␈α∞of
␈↓ α∧␈↓motion␈αin␈α a␈αsuitable␈α form,␈αwhereas␈α it␈αmay␈α be␈αfeasible␈α to␈αmake␈α a␈αprediction␈αof␈α the␈αeffects␈αof
␈↓ α∧␈↓the  beliefs we ascribe to  the program without any computer at all.

␈↓ α∧␈↓        #.␈α
  The␈αbelief␈α
and␈α
goal␈αstructures␈α
we␈αascribe␈α
to␈α
the␈αprogram␈α
may␈αbe␈α
easier␈α
to␈αunderstand
␈↓ α∧␈↓than the details of  program as expressed in its listing.

␈↓ α∧␈↓        #.␈α∞The␈α∞belief␈α∞and␈α∂goal␈α∞structure␈α∞is␈α∞likely␈α∂to␈α∞be␈α∞close␈α∞to␈α∂the␈α∞structure␈α∞the␈α∞designer␈α∂ of␈α∞the
␈↓ α∧␈↓program␈α
had␈α
in␈α
 mind,␈α∞and␈α
 it␈α
may␈α
 be␈α∞easier␈α
to␈α
debug␈α
the␈α∞program␈α
in␈α
terms␈α
 of␈α∞this␈α
structure
␈↓ α∧␈↓than␈α
directly␈α
from␈α
 the␈α∞listing.␈α
  In␈α
 fact,␈α
it␈α∞ is␈α
often␈α
 possible␈α
for␈α∞ someone␈α
to␈α
correct␈α
a␈α∞fault␈α
by
␈↓ α∧␈↓reasoning␈α in␈αgeneral␈αterms␈αabout␈αthe␈α information␈αin␈αa␈αprogram␈αor␈αmachine,␈αdiagnosing␈α what␈αis
␈↓ α∧␈↓wrong␈α⊃as␈α⊂a␈α⊃false␈α⊃belief,␈α⊂and␈α⊃  looking␈α⊂ at␈α⊃ the␈α⊃ details␈α⊂ of␈α⊃  the␈α⊂ program␈α⊃ or␈α⊃ machine␈α⊂ only
␈↓ α∧␈↓sufficiently␈α∞to␈α∞determine␈α∞how␈α∞ the␈α∞false␈α∞belief␈α∞is␈α∞represented␈α∞ and␈α∞what␈α∞mechanism␈α∞caused␈α∞it␈α∞to
␈↓ α∧␈↓arise.

␈↓ α∧␈↓        This␈α
 kind␈α of␈α
 teleological␈α analysis␈α
 is␈α often␈α
 useful␈α in␈α
understanding␈α
natural␈αorganisms
␈↓ α∧␈↓as␈αwell␈αas␈αmachines.␈α Here␈α evolution␈αtakes␈α the␈αplace␈α of␈α design␈αand␈α we␈α often␈αunderstand␈α the
␈↓ α∧␈↓function␈α⊃performed␈α⊃by␈α⊃ an␈α⊃organ␈α⊃before␈α⊃we␈α⊃ understand␈α⊃its␈α⊃detailed␈α⊃physiology.␈α⊂ Teleological



␈↓ α∧␈↓analysis␈α is␈α applicable␈αto␈α psychological␈α and␈α social␈αphenomena␈αin␈α so␈αfar␈αas␈α these␈αare␈αdesigned
␈↓ α∧␈↓or have been  subject to selection.

␈↓ α∧␈↓        (However,␈α teleological␈αanalysis␈αfails␈αwhen␈αapplied␈αto␈αaspects␈αof␈αnature␈αwhich␈α have␈αneither
␈↓ α∧␈↓been␈α∞ designed␈α∂nor␈α∞produced␈α∞by␈α∂ natural␈α∞selection␈α∞from␈α∂a␈α∞population.␈α∞  Much␈α∂medieval␈α∞science
␈↓ α∧␈↓was␈α
based␈αon␈α
the␈αJudeo-Christian-Moslem␈α
religious␈αhypothesis␈α
that␈α the␈α
details␈αof␈α
 the␈αworld␈α
were
␈↓ α∧␈↓designed␈α
by␈α
God␈α
 for␈α
the␈α
benefit␈α
of␈αman.␈α
  The␈α
strong␈α
form␈α
of␈α
 this␈α
hypothesis␈α
was␈αabandoned
␈↓ α∧␈↓at␈αthe␈αtime␈αof␈α Galileo␈αand␈α
Newton␈αbut␈αoccasionally␈αrecurs.␈α  Barry␈αCommoner's␈α(1972)␈α
axiom␈αof
␈↓ α∧␈↓ecology␈α
"Nature␈αknows␈α
best"␈α seems␈α
to␈αbe␈α
mistakenly␈α
based␈αon␈α
 the␈αnotion␈α
that␈αnature␈α
 as␈αa␈α
 whole
␈↓ α∧␈↓is the  result of  an evolutionary  process that selected the "best nature").

␈↓ α∧␈↓        Computers␈αgive␈αrise␈αto␈αnumerous␈αexamples␈αof␈αbuilding␈αa␈αhigher␈αstructure␈αon␈αthe␈αbasis␈αof␈αa
␈↓ α∧␈↓lower␈α
and␈α
conducting␈α
subsequent␈α
analyses␈α
using␈α
the␈α
higher␈α
structure.␈α
 The␈α
geometry␈α
of␈αthe␈α
electric
␈↓ α∧␈↓fields␈α
in␈αa␈α
transistor␈α
and␈αits␈α
chemical␈α
composition␈αgive␈α
rise␈αto␈α
its␈α
properties␈αas␈α
an␈α
electric␈αcircuit
␈↓ α∧␈↓element.␈α∂ Transistors␈α∞are␈α∂combined␈α∞in␈α∂small␈α∞circuit␈α∂and␈α∞powered␈α∂in␈α∞standard␈α∂ways␈α∞to␈α∂make␈α∞up
␈↓ α∧␈↓logical␈α∩elements␈α∩such␈α∩as␈α∩ANDs,␈α∩ORs,␈α∩NOTs␈α∩and␈α∩flip-flops.␈α∩ The␈α∩computer␈α∩designer␈α⊃designs
␈↓ α∧␈↓hardware␈αwith␈αthese␈αlogical␈αelements␈αto␈αobey␈αa␈αdesired␈αorder␈αcode;␈αin␈αdoing␈αthis␈αhe␈αtries␈αto␈αavoid
␈↓ α∧␈↓having␈αto␈αconsider␈αthe␈αproperties␈αof␈αthe␈αtransistors␈αas␈αcircuit␈αelements.␈α The␈αdesigner␈αof␈αa␈αhigher
␈↓ α∧␈↓level␈α∂language␈α∞works␈α∂in␈α∞terms␈α∂of␈α∞the␈α∂order␈α∞code␈α∂and␈α∞hopefully␈α∂doesn't␈α∞have␈α∂to␈α∞know␈α∂abut␈α∞the
␈↓ α∧␈↓logic; the user of his higher order language doesn't have to know about the order code.

␈↓ α∧␈↓        In␈α⊂the␈α⊂above␈α⊂cases,␈α⊂users␈α⊂of␈α⊂the␈α⊂higher␈α⊂level␈α⊂can␈α⊂completely␈α⊂ignore␈α⊂the␈α⊂lower␈α⊃level,␈α⊂the
␈↓ α∧␈↓behavior␈α∞of␈α∞the␈α∞higher␈α∂level␈α∞system␈α∞is␈α∞completely␈α∞determined␈α∂by␈α∞the␈α∞values␈α∞of␈α∞the␈α∂higher␈α∞level
␈↓ α∧␈↓variables,␈αe.g.␈α in␈αorder␈αto␈αdetermine␈αthe␈αoutcome␈αof␈αa␈αcomputer␈αprogram,␈αone␈αshould␈αnot␈αhave␈αto
␈↓ α∧␈↓look␈αat␈αthe␈αstates␈αof␈αflip-flops.␈α However,␈αwhen␈αwe␈αascribe␈αmental␈αstructure␈αto␈αhumans␈αor␈αgoals␈αto
␈↓ α∧␈↓society,␈α∪we␈α∪always␈α∪get␈α∪highly␈α∪incomplete␈α∩systems;␈α∪the␈α∪higher␈α∪level␈α∪behavior␈α∪cannot␈α∪be␈α∩fully
␈↓ α∧␈↓predicted␈αfrom␈αhigher␈αlevel␈αobservations␈αand␈αhigher␈αlevel␈α"laws"␈αeven␈αwhen␈αthe␈αunderlying␈αlower
␈↓ α∧␈↓level behavior is determinate.

␈↓ α∧␈↓EXAMPLES

␈↓ α∧␈↓        Let␈α⊂us␈α⊂ consider␈α⊂some␈α⊂ examples␈α⊂of␈α⊃machines␈α⊂ and␈α⊂programs␈α⊂ to␈α⊂which␈α⊂we␈α⊃may␈α⊂ascribe
␈↓ α∧␈↓belief and goal structures.

␈↓ α∧␈↓        1.␈α∂ Thermostats.␈α⊂  Ascribing␈α∂beliefs␈α⊂to␈α∂simple␈α⊂thermostats␈α∂is␈α⊂not␈α∂really␈α⊂necessary,␈α∂because
␈↓ α∧␈↓their␈α⊃operation␈α⊃can␈α⊃be␈α⊃well␈α⊃ understood␈α⊃without␈α⊃it.␈α⊃ However,␈α⊃their␈α⊃very␈α⊃simplicity␈α⊃ makes␈α⊂it
␈↓ α∧␈↓clearer␈αwhat␈α
is␈αinvolved␈α in␈α
the␈α ascription,␈αand␈α
 we␈αmaintain␈α(to␈α
 some␈αextent␈α as␈α
a␈αprovocation
␈↓ α∧␈↓aimed␈α
 at␈α
 those␈α
who␈α
 regard␈α
 attribution␈α
of␈α
 beliefs␈α
 to␈α
machines␈α
 as␈α
mere␈α intellectual␈α
 sloppiness)
␈↓ α∧␈↓that  the ascription  is legitimate even if unnecessary.

␈↓ α∧␈↓        First␈α let␈αus␈αconsider␈αa␈α
simple␈α thermostat␈αthat␈αturns␈αon␈α
the␈αheat␈αwhen␈αthe␈α temperature␈αis␈α
a
␈↓ α∧␈↓degree␈α
 above␈α
the␈αtemperature␈α
set␈α
 on␈α
the␈αthermostat,␈α
 turns␈α
on␈α
the␈αheat␈α
 when␈α
the␈αtemperature␈α
 is
␈↓ α∧␈↓a␈αdegree␈α
below␈αthe␈αdesired␈α
 temperature,␈αand␈αleaves␈α
 the␈αheat␈α
as␈α is␈αwhen␈α
 the␈αtemperature␈α is␈α
  in
␈↓ α∧␈↓the␈α
 two␈α
  degree␈α
 range␈α around␈α
  the␈α
 desired␈α
temperature.␈αThe␈α
simplest␈α
belief␈α
predicate␈α␈↓↓B(s,p)␈↓
␈↓ α∧␈↓ascribes␈α∞belief␈α∞to␈α∞only␈α
two␈α∞sentences:␈α∞ "The␈α∞room␈α
is␈α∞too␈α∞cold"␈α∞ and␈α
"The␈α∞room␈α∞is␈α∞ too␈α∞hot",␈α
and
␈↓ α∧␈↓these␈α beliefs␈αare␈αassigned␈αto␈αstates␈αof␈α the␈αthermostat␈αso␈αthat␈α in␈α the␈α one␈α degree␈αrange,␈α neither
␈↓ α∧␈↓is␈α⊂ believed.␈α⊂   When␈α⊂the␈α⊂thermostat␈α⊂believes␈α⊂ the␈α⊂ room␈α⊂is␈α⊂ too␈α⊂cold␈α⊂ or␈α⊂too␈α⊂ hot␈α⊂ it␈α⊂sends␈α∂ a



␈↓ α∧␈↓message␈αto␈αthat␈αeffect␈αto␈αthe␈αfurnace.␈αA␈αslightly␈αmore␈αcomplex␈αbelief␈αpredicate␈αcould␈αalso␈α be␈αused
␈↓ α∧␈↓in␈α which␈αthe␈αthermostat␈α has␈αa␈α belief␈αabout␈αwhat␈αthe␈αtemperature␈αshould␈αbe␈αand␈αanother␈αbelief
␈↓ α∧␈↓about␈αwhat␈αit␈αis.␈α  It␈αis␈αnot␈αclear␈α which␈αis␈αbetter,␈αbut␈αif␈α we␈αwished␈αto␈αconsider␈αpossible␈αerrors␈αin
␈↓ α∧␈↓the␈α∃thermometer,␈α∃the␈α∃then␈α∃we␈α∀would␈α∃have␈α∃to␈α∃be␈α∃able␈α∀to␈α∃ascribe␈α∃beliefs␈α∃ about␈α∃what␈α∀the
␈↓ α∧␈↓temperature␈α
is.␈α
We␈α
 do␈α
not␈α
ascribe␈α
to␈α
 it␈α
any␈α
other␈α
 beliefs;␈α
it␈α
has␈α
 no␈α
opinion␈α
even␈α about␈α
whether
␈↓ α∧␈↓the␈αheat␈αis␈αon␈αor␈αoff␈αor␈αabout␈αthe␈αweather␈αor␈αabout␈αwho␈αwon␈αthe␈αbattle␈αof␈αWaterloo.␈α   Moreover,
␈↓ α∧␈↓it  has  no introspective  beliefs,  i.e.   it doesn't believe that it believes the room is too hot.

␈↓ α∧␈↓        The␈α∞temperature␈α∞control␈α∞system␈α∞in␈α∞ my␈α∞house␈α∞may␈α∞be␈α∞ described␈α∞as␈α∂ follows:␈α∞Thermostats
␈↓ α∧␈↓upstairs␈αand␈α downstairs␈α
 tell␈αthe␈α central␈α
system␈α to␈αturn␈α on␈α
or␈αshut␈α off␈α
hot␈α water␈αflow␈α to␈α
these
␈↓ α∧␈↓areas.␈α  A␈αcentral␈αwater-temperature␈αthermostat␈αtells␈αthe␈αfurnace␈αto␈αturn␈α on␈αor␈αoff␈α thus␈α keeping
␈↓ α∧␈↓the␈α central␈α hot␈α water␈α reservoir␈αat␈α the␈α right␈αtemperture.␈α Recently␈αit␈αwas␈αtoo␈αhot␈αupstairs,␈αand
␈↓ α∧␈↓the␈α⊂question␈α⊂arose␈α⊃as␈α⊂to␈α⊂whether␈α⊃the␈α⊂upstairs␈α⊂thermostat␈α⊂mistakenly␈α⊃ ␈↓↓believed␈↓␈α⊂it␈α⊂was␈α⊃too␈α⊂ cold
␈↓ α∧␈↓upstairs␈α∞ or␈α
 whether␈α∞ the␈α
furnace␈α∞ thermostat␈α∞ mistakenly␈α
␈↓↓believed␈α∞␈↓␈α
the␈α∞ water␈α
was␈α∞too␈α∞ cold.␈α
 It
␈↓ α∧␈↓turned␈αout␈αthat␈α neither␈αmistake␈αwas␈αmade;␈αthe␈αdownstairs␈αcontroller␈α␈↓↓tried␈↓␈αto␈αturn␈αoff␈αthe␈αflow␈α of
␈↓ α∧␈↓water␈α⊂but␈α∂ ␈↓↓couldn't␈↓,␈α⊂because␈α∂the␈α⊂ valve␈α∂was␈α⊂ stuck.␈α∂ The␈α⊂plumber␈α∂came␈α⊂ once␈α∂ and␈α⊂found␈α∂ the
␈↓ α∧␈↓trouble,␈α∩ and␈α∩ came␈α∪again␈α∩ when␈α∩ a␈α∪replacement␈α∩valve␈α∩ was␈α∪ordered.␈α∩  Since␈α∩the␈α∪services␈α∩of
␈↓ α∧␈↓plumbers␈α
are␈α
increasingly␈α
 expensive,␈α
and␈α
 microcomputers␈α
are␈α
 increasingly␈α
cheap,␈α
one␈α
is␈α
led␈αto
␈↓ α∧␈↓design␈α
a␈α
temperature␈α
control␈α
system␈α
that␈α
would␈α
␈↓↓know␈↓␈α
a␈α
lot␈α
more␈α
about␈α
the␈α
thermal␈α
 state␈α∞of␈α
the
␈↓ α∧␈↓house and its own state of health.

␈↓ α∧␈↓        In␈α∞the␈α∞first␈α∂ place,␈α∞while␈α∞the␈α∂ system␈α∞␈↓↓couldn't␈↓␈α∞turn␈α∂ off␈α∞the␈α∞flow␈α∂of␈α∞hot␈α∞water␈α∂ upstairs,␈α∞it
␈↓ α∧␈↓there␈αis␈αno␈αreason␈α to␈αascribe␈αto␈αit␈αthe␈α ␈↓↓knowledge␈↓␈αthat␈αit␈αcouldn't,␈αand␈α ␈↓↓a␈αfortiori␈↓␈αit␈αhad␈αno␈αability
␈↓ α∧␈↓to␈α
␈↓↓communicate␈↓␈αthis␈α
␈↓↓fact␈↓␈α
 or␈αto␈α
take␈α
it␈αinto␈α
 account␈αin␈α
controlling␈α
the␈αsystem.␈α
 A␈α
more␈αadvanced
␈↓ α∧␈↓system␈α∪would␈α∪know␈α∪whether␈α∩the␈α∪ ␈↓↓actions␈↓␈α∪it␈α∪␈↓↓attempted␈↓␈α∩succeeded,␈α∪and␈α∪it␈α∪would␈α∩communicate
␈↓ α∧␈↓failures␈αand␈αadapt␈α to␈α
them.␈α (We␈α adapted␈αto␈α
the␈αfailure␈α by␈αturning␈α
off␈α the␈αwhole␈α system␈α
 until
␈↓ α∧␈↓the␈α∞ whole␈α∂ house␈α∞cooled␈α∂ off␈α∞and␈α∂ then␈α∞letting␈α∂the␈α∞two␈α∂parts␈α∞warm␈α∂up␈α∞ together.␈α∂ The␈α∞present
␈↓ α∧␈↓system has  the ␈↓↓physical  capability␈↓  of  doing  this  even   if  it  hasn't  the ␈↓↓knowledge␈↓ or the ␈↓↓will␈↓.

␈↓ α∧␈↓        2.␈α
The␈α"Life"␈α
cellular␈α
automaton.␈α This␈α
example,␈αwhich␈α
was␈α
invented␈αfor␈α
quite␈α
a␈αdifferent
␈↓ α∧␈↓purpose␈α⊃by␈α⊃John␈α⊃Conway,␈α⊃can␈α⊃be␈α⊃used␈α⊃as␈α⊃a␈α⊃simplified␈α⊃model␈α⊃of␈α⊃the␈α⊃world␈α⊃as␈α⊃a␈α∩whole␈α⊃and
␈↓ α∧␈↓illustrates␈α⊃some␈α⊃phenomena␈α⊃that␈α∩occur␈α⊃in␈α⊃the␈α⊃real␈α⊃world,␈α∩but␈α⊃without␈α⊃all␈α⊃of␈α⊃the␈α∩real␈α⊃world's
␈↓ α∧␈↓complexity.

␈↓ α∧␈↓        At␈α
each␈α
point␈α
␈↓↓(x,y)␈↓␈α
in␈α
the␈α
plane␈αwith␈α
integer␈α
co-ordinates␈α
an␈α
automaton␈α
with␈α
two␈αpossible
␈↓ α∧␈↓states␈α0␈αand␈α1␈αis␈αattached.␈α The␈αstate␈αof␈αeach␈αautomaton␈αat␈αtime␈α␈↓↓t+1␈↓␈αis␈αdetermined␈αfrom␈αits␈αstate␈α
at
␈↓ α∧␈↓time␈α
␈↓↓t␈↓␈α
and␈α
the␈αstates␈α
of␈α
its␈α
eight␈αneighbors␈α
as␈α
follows:␈α
A␈αpoint␈α
whose␈α
state␈α
is␈α0␈α
will␈α
become␈α
1␈αif
␈↓ α∧␈↓exactly␈αthree␈αof␈α
its␈αneighbors␈αare␈αin␈α
state␈α1.␈α A␈αpoint␈α
whose␈αstate␈αis␈α1␈α
will␈αremain␈α1␈αif␈α
2␈αor␈α3␈αof␈α
its
␈↓ α∧␈↓neighbors are in state 1.

␈↓ α∧␈↓        The␈αoriginal␈αquestion␈αasked␈αby␈αConway␈α was␈αwhether␈αan␈αinitial␈αconfiguration␈α with␈αonly␈αa
␈↓ α∧␈↓finite␈α set␈αof␈α1's␈αwould␈α remain␈αthat␈αway.␈α After␈αConway␈αand␈αothers␈αdiscovered␈αinfinitely␈αgrowing
␈↓ α∧␈↓configurations,␈αthe␈αquestions␈αchanged,␈α and␈αit␈αwas␈αeventually␈αproved␈α that␈αthe␈α"life"␈αautomaton␈αis
␈↓ α∧␈↓universal␈αfor␈αcomputation,␈αi.e.␈αan␈αinitial␈αconfiguration␈αcan␈αbe␈αfound␈αthat␈αwill␈α interpret␈αa␈αsuitably
␈↓ α∧␈↓encoded␈α∞data␈α
 in␈α∞another␈α∞part␈α
 of␈α∞the␈α
plane␈α∞ as␈α∞an␈α
 arbitrary␈α∞computer␈α
 program␈α∞and␈α∞ its␈α
data.
␈↓ α∧␈↓Moreover,␈α
 it␈α
 was␈α
 discovered␈α
 that␈α
 such␈α
 computers␈α
 can␈α
 be␈α
  made␈α
self-reproducing␈α
 as␈α well,
␈↓ α∧␈↓i.e.␈α
   they␈α can␈α
 make␈α new␈α
 copies␈α of␈α
themselves␈αif␈α
suitably␈α programmed,␈α
and␈αthese␈α
copies␈αcan
␈↓ α∧␈↓also compute and reproduce.



␈↓ α∧␈↓        Now␈α
imagine␈α
such␈α
a␈α
computer␈α
constructed␈α
of␈α
"life"␈α
elements.␈α
 Among␈α
the␈α
things␈α
it␈α∞can␈α
be
␈↓ α∧␈↓programmed␈α∞to␈α∂do␈α∞is␈α∞to␈α∂simulate␈α∞the␈α∞"life"␈α∂automaton␈α∞system,␈α∞i.e.␈α∂it␈α∞can␈α∞know␈α∂the␈α∞fundamental
␈↓ α∧␈↓physics␈αof␈αits␈αworld.␈α
 Moreover,␈αit␈αcan␈αbe␈αtold␈α
the␈αinitial␈αconfiguration␈αof␈α
its␈αworld␈αby␈αstoring␈αin␈α
a
␈↓ α∧␈↓strip␈α∂of␈α∂data␈α∂a␈α∂list␈α∂of␈α∞all␈α∂cells␈α∂outside␈α∂the␈α∂strip␈α∂that␈α∂are␈α∞in␈α∂state␈α∂1.␈α∂ It␈α∂doesn't␈α∂need␈α∂a␈α∞separate
␈↓ α∧␈↓description␈α∞of␈α∞the␈α∞states␈α∞of␈α∞the␈α∞cells␈α∞inside␈α
the␈α∞strip,␈α∞since␈α∞it␈α∞knows␈α∞the␈α∞code.␈α∞ Therefore,␈α∞it␈α
can
␈↓ α∧␈↓simulate␈α
its␈α
world␈α∞and␈α
answer␈α
any␈α∞question␈α
about␈α
the␈α
state␈α∞of␈α
the␈α
world␈α∞at␈α
a␈α
definite␈α∞time.␈α
 Of
␈↓ α∧␈↓course, it must compute so slowly that it can never predict its own future.

␈↓ α∧␈↓        Now␈αsuppose␈αwe␈αhave␈αseveral␈αsuch␈α"computers"␈αthat␈αinteract␈αwith␈αeach␈αother,␈αbut␈αeach␈αone
␈↓ α∧␈↓knows␈αthe␈αentire␈αinitial␈αconfiguration␈αand␈αthe␈α"law␈αof␈αmotion"␈αof␈α"life".␈α Suppose␈αfurther␈αthat␈αwe
␈↓ α∧␈↓program␈αeach␈αof␈αthem␈αto␈αtry␈αto␈αachieve␈α
certain␈αgoals␈αand␈αthat␈αto␈αdo␈αthis␈αthey␈α
must␈αcommunicate
␈↓ α∧␈↓with␈αeach␈αother␈α
and␈αco-operate␈αand␈αcompete.␈α
 It␈αseems␈αevident␈α
that␈αeffective␈αbehavior␈αwill␈α
require
␈↓ α∧␈↓that␈αa␈αcomputer␈αascribe␈αbeliefs␈αand␈αpurposes␈αto␈αthe␈αothers␈αeven␈αthough␈αit␈αknows␈αthe␈αfundamental
␈↓ α∧␈↓physics␈αof␈αthe␈αworld␈αand␈αcould␈α"in␈αprinciple"␈αpredict␈αthe␈αother's␈αbehavior␈αfrom␈αany␈αknown␈αinitial
␈↓ α∧␈↓state.␈α It␈αwill␈αhave␈αto␈αformulate␈αtheories␈αabout␈αhow␈αthese␈αbeliefs␈αarise␈αand␈αhow␈αthey␈αaffect␈αaction.
␈↓ α∧␈↓How␈αthese␈αbeliefs␈αand␈αwants␈α␈↓↓reduce␈↓␈αto␈αconfigurations␈αof␈α0's␈αand␈α1's␈αin␈αthe␈α"life"␈αautomata␈αmay␈αbe
␈↓ α∧␈↓neither␈α⊂useful␈α⊂nor␈α⊂easily␈α⊂determined␈α⊂either␈α⊂by␈α⊂the␈α⊂computer␈α⊂itself␈α⊂or␈α⊂by␈α⊂ourselves␈α⊂as␈α⊂external
␈↓ α∧␈↓observers.

␈↓ α∧␈↓        The␈α⊂moral␈α⊂of␈α⊂this␈α⊂example␈α⊂is␈α⊂to␈α⊃emphasize␈α⊂that␈α⊂the␈α⊂problem␈α⊂of␈α⊂the␈α⊂relation␈α⊃of␈α⊂mental
␈↓ α∧␈↓qualities to physical states arises in physical systems of the most determinate kind possible.

␈↓ α∧␈↓        3.␈α⊂Computer␈α∂time-sharing␈α⊂systems.␈α∂ These␈α⊂are␈α∂quite␈α⊂complicated␈α∂computer␈α⊂programs␈α∂that
␈↓ α∧␈↓share computer time among users.


␈↓ α∧␈↓␈↓ ∧;INTROSPECTION AND SELF-KNOWLEDGE

␈↓ α∧␈↓        We␈α say␈αthat␈α a␈α machine␈αintrospects␈α when␈αit␈α comes␈α to␈αhave␈αbeliefs␈αabout␈αits␈αown␈αmental
␈↓ α∧␈↓state.␈α∞ A␈α∞simple␈α∞form␈α∂of␈α∞ introspection␈α∞takes␈α∞ place␈α∂ when␈α∞ a␈α∞ program␈α∞ determines␈α∂ whether␈α∞ it
␈↓ α∧␈↓has␈α certain␈αinformation␈α and␈αif␈αnot␈αasks␈αfor␈αit.␈α  Often␈αan␈αoperating␈αsystem␈αwill␈αcompute␈αa␈αcheck
␈↓ α∧␈↓sum␈αof␈αitself␈α
 every␈αfew␈αminutes␈α
 to␈αverify␈αthat␈α it␈α
hasn't␈αbeen␈αchanged␈α
by␈αa␈αsoftware␈αor␈α
hardware
␈↓ α∧␈↓malfunction.

␈↓ α∧␈↓        In␈α∞principle,␈α∂ introspection␈α∞is␈α∞easier␈α∂ for␈α∞computer␈α∞programs␈α∂than␈α∞for␈α∞people,␈α∂because␈α∞the
␈↓ α∧␈↓entire␈α∂memory␈α⊂in␈α∂which␈α⊂programs␈α∂and␈α⊂data␈α∂are␈α⊂stored␈α∂is␈α⊂available␈α∂for␈α⊂inspection.␈α∂  In␈α⊂fact,␈α∂a
␈↓ α∧␈↓computer␈αprogram␈αcan␈α be␈α made␈αto␈α predict␈αhow␈α it␈α would␈αreact␈α to␈α particular␈αinputs␈αprovided
␈↓ α∧␈↓it␈αhas␈αenough␈αfree␈αstorage␈α
to␈αperform␈αthe␈αcalculation.␈α This␈α
situation␈αsmells␈αof␈αparadox,␈αand␈α
there
␈↓ α∧␈↓is␈αone.␈α Namely,␈αif␈αa␈αprogram␈αcould␈αpredict␈αits␈αown␈αactions␈αin␈αless␈αtime␈αthan␈αit␈αtakes␈αto␈αcarry␈αout
␈↓ α∧␈↓the␈α∂action,␈α∂it␈α∂could␈α∂refuse␈α∂ to␈α∂do␈α∂what␈α∂it␈α∂has␈α∂predicted␈α∂ for␈α∂itself.␈α∂ This␈α∂only␈α∂ shows␈α∂that␈α∞self-
␈↓ α∧␈↓simulation  is necessarily a  slow process which is not surprising.

␈↓ α∧␈↓        However,␈α⊂  present␈α⊂   programs␈α⊂   do␈α⊂   little␈α∂   interesting␈α⊂introspection.␈α⊂  This␈α⊂is␈α⊂just␈α∂ a
␈↓ α∧␈↓matter␈αof␈α the␈α
undeveloped␈α state␈αof␈α
artificial␈α intelligence;␈αprogrammers␈α
 don't␈αyet␈αknow␈α
 how␈αto
␈↓ α∧␈↓make a computer program look at itself in a useful way.

␈↓ α∧␈↓John McCarthy Artificial Intelligence Laboratory Stanford University Stanford, California 94305